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Surveys continue to be one of the primary research methods in social science research, as 
they have been useful for exploring subjects ranging from attitudes and intentions to motivations 
and behaviors, to name but a few. Notwithstanding, response rates in survey research continue to 
decline (Rogelberg, Conway, Sederburg, Spitzmuller, Aziz, & Knight, 2003) despite the 
development of more systematic procedures to optimize study participation like Dillman’s 
Tailored Design Method (Dillman, 2007). Web-based surveys have drawn researcher attention, 
too, because they seem to be a relatively fast and inexpensive means to conduct research, yet 
often they are plagued by even lower response rates than conducted via traditional surveys 
(Sheehan & Hoy, 1999; Simsek & Veiga, 2001). 

According to Rogelberg et al. (2003), low response rates in survey research can be 
problematic for three important reasons. First, lower response rates mean fewer participants, 
which reduces statistical power and prevents the use of certain statistical procedures. Second, 
low response rates can reduce the perceived credibility of the study’s findings. In general, when 
faced with unfavorable results, survey sponsors cite low response rates as “the issue” instead of 
plausible alternatives. Third, low response rates can generate biased samples where study 
participants are systematically different from nonrespondents. Each of the aforementioned issues 
can limit unnecessarily the generalizability of the findings to the research population. In this 
perspective, I explore the third issue of nonresponse bias because it is probably the most serious 
of these concerns (Rogelberg et al., 2003) and because of its cutting-edge implications for the 
application of social science research in fields like adult education and human resource 
development (Bartlett, Reio, & Bartlett, in press). By understanding how survey respondents 
compare with nonrespondents on key demographic and research variables, researchers can 
generalize their findings more accurately and confidently. 

Rogelberg and Luong (1998) state that “Nonresponse bias occurs when the individuals 
responding to a survey differ from nonrespondents on variables relevant to the survey topic” (p. 
60). Quantitatively speaking, nonresponse bias is defined as: 
nonresponse bias = NR (X res - X non ) 

NR = proportion of nonrespondents in sample 
X res = Mean of the respondents 

X n on = Mean of the nonrespondents 

I must point out that nonresponse in survey research does not necessarily mean that there 
is nonresponse bias. Strangely enough, a sample may not have nonresponse bias even though its 
response rate is very low. For example, at an organizational training center where morale may be 
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dismal, the mean of respondents’ morale might be 4.0 with a 10% response rate; yet the mean of 
nonrespondents, if collected, could be 4.0 as well. Thus, there is no evidence of nonresponse 
bias. However, if the difference between the two group means is substantial, nonresponse bias is 
evident even though the response rate is very high (Rogelberg & Luong, 1998). Each finding 
obviously would have implications for appropriately interpreting and generalizing the findings to 
the research population. 

Survey nonresponse is typically classified into four discrete categories: inaccessibility, 
inability, carelessness, and noncompliance (Rogelberg et ah, 2003). Inaccessibility means that 
the respondent never received the survey, whereas inability means that the respondent was not 
able to respond due to illness or the like. Carelessness means the respondent misplaced the 
survey, whereas active noncompliance means the respondent consciously decided not to 
participate in the study. Active noncompliance in particular has received the most research 
attention. Active noncompliance tends to occur because the prospective participant is not 
interested in the topic. Assessing interest levels and then testing for possible relationships 
between interest and the research variable responses may provide valuable insights into the 
extent to which interest is associated with the variable in question. In addition, research has 
demonstrated limited empirical support for seeing personality (responsible, intellectual) and 
sociodemographic (age, gender, socioeconomic status) variables as reasons for noncompliance. 
Interestingly, McFarlane, Olmstead, Murphy, and Hill (2007) discovered that male responders 
tended to be early responders, but possible gender bias was reduced through repeated mailings. 

I would like to introduce the notion that it is possible active noncompliance occurs for 
other reasons, such as fear of being exposed in situations marked by asymmetry of power or 
when the prospective respondent could be easily identified because of his or her ethnicity or job 
title. Further, some even may object to the perceived undue influence that surveys have on news 
coverage, politics, and public policy and, thus, choose not to participate. For example, exit polls 
at polling sites during elections have received increased scrutiny in recent years. It is intriguing 
that very little research has addressed this part of the active noncompliance issue systematically. 

In general, the most salient reasons for noncompliance have been lack of interest and 
lower education levels (Rogelberg & Fuong, 1998). Thus, researchers need to acknowledge these 
bias possibilities in their study designs by developing a number of strategies to optimize response 
rates and thereby reduce the likelihood of nonresponse bias. 

Although beyond the scope of this article, Dillman’s (2007) Tailored Design Method has 
particular promise for improving survey response rates. In essence, through a systematic effort of 
survey personalization, pre-notification, timed mailings, and follow-ups, response rates for 
studies using Dillman’s method can be as high as 85% for mailed surveys and 55% for web- 
based surveys. Telephone survey response rates closely mirror mail response rates when 
employing the tailored design method. Response rates can be increased further with token 
incentives; but this may introduce another source of bias because those induced to participate 
may differ significantly from nonrespondents on the research variable. 

A number of strategies can be used to detect the presence of nonresponse bias after study 
completion. Rogelberg et al. (2003) describes four approaches: archival, follow-up, wave, and 
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intention. The archival approach constitutes comparing respondents and the population on the 
research variables. For example, personnel records li nk ed to respondents can be directly 
compared to those of nonrespondents. Archival data tends to be demographic and thus has 
limited utility. The follow-up approach involves contacting a random sample of nonrespondents 
via phone and having them complete a shortened version of the survey. Respondents’ and 
nonrespondents’ scores are subsequently compared to test for possible differences; however, this 
method may be subject to social desirability responses. The wave approach compares the scores 
of those who met the survey completion deadline against those of respondents who were late 
completers. The assumption is that late responders are more like nonrespondents than 
respondents. The major limitation of this approach is that actual nonrespondents are not studied. 
Finally, the intention approach entails asking potential respondents about their intention to 
complete the survey and then surveying them anyway. Researchers then compare the scores of 
those who intended to participate and those who did not. As with the wave approach, actual 
nonrespondents are not examined. 

The issue of nonresponse is an insidious one. Researchers and practitioners alike must be 
aware that not examining survey results for possible nonresponse bias limits their 
generalizability to the research population. The problem is that much of the published social 
science literature, to its possible detriment, overlooks the topic. The fields of adult education and 
human resource development certainly can benefit from carefully addressing this issue in 
published research. Besides taking appropriate steps to facilitate optimal response rates, we can 
minimize the likelihood of nonresponse bias by specifically testing for it in a number of 
meaningful ways (Rogelberg et ah, 2003). Although the archival, follow-up, wave, and intention 
approaches have limitations, they can at the very least provide a preliminary sense of the data’s 
representativeness. If testing for nonresponse bias is not possible, the researcher should 
acknowledge at least the possibility of nonresponse bias. Productively addressing possible 
nonresponse bias in survey research can serve only to increase its utility and credibility. 
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